Skip to content

Add a sparse keyword to reductions along a dimension and reduce column-range views through the sparse kernels - #798

Open
ViralBShah wants to merge 5 commits into
mainfrom
vs/sparse-reductions
Open

ViralBShah wants to merge 5 commits into
mainfrom
vs/sparse-reductions

Conversation

@ViralBShah

@ViralBShah ViralBShah commented Sep 9, 2026

Copy link
Copy Markdown
Member

Fixes #43. Fixes #377.

Issue. sum(A; dims = 2) on a hypersparse matrix allocates a dense result proportional to size(A, 1) however few entries A stores (#43). Reductions of a column-range view went through Base's element-wise fallback and returned a sparse result, since Base's similar on a SubArray defers to the parent (#377).

A = sparse([1, 10^7], [1, 150], [1.0, 2.0], 10^7, 150)
@time sum(A; dims = 2)            # 15 ms, 153 MiB, for two stored entries
B = sprand(101, 99, 0.01)
sum(view(B, :, 5:11); dims = 1)   # SparseMatrixCSC, element by element; the copy gives a Matrix

Fix. The default stays a dense Matrix, as for dense input (see the discussion below). A sparse result is opt-in: sum(A; dims = 2, sparse = true), and likewise for prod, maximum, minimum, count, any, all and mapreduce, with init where they accept it. It stores an entry for each row or column that stores one, or for every slice when an unstored slice reduces to something nonzero, with the element type of the dense result. Column-range views reduce through the sparse kernels and return the same Matrix as their copy; nnz of such a view is O(1).

Mechanism. Base forwards unknown keywords from sum, prod, maximum and minimum to mapreduce, so one mapreduce method carries the keyword; any, all and count get their own. Slices are seeded as Base seeds the dense result, with init or mapreduce_first, and unstored entries are folded in through the existing _mapreducezeros. Element type and the value of an empty slice come from Base's reducedim_init on a stand-in. Row reductions build the result column by column; column reductions use a dense workspace, or sort the stored entries by row when there are fewer than m / 8 of them.

Dispatch. New sparse-keyword methods for Base.mapreduce, any, all and count on SparseMatrixCSCUnion; the existing sparse reduction methods widen from AbstractSparseMatrixCSC to SparseMatrixCSCUnion. Behaviour change: reductions along a dimension of a column-range view now return a Matrix. extrema with sparse = true throws an ArgumentError, since a tuple has no zero.

Measured on 1.14-DEV, min over samples:

case dense default sparse = true
sum(A; dims=2), 10^7 x 150, 151 entries 15.1 ms, 153 MiB 2.4 µs, 16 KiB
sum(B; dims=1), 10^4 x 10^4, 1e-2 0.16 ms 0.19 ms
sum(B; dims=2), 10^4 x 10^4, 1e-2 1.26 ms 1.57 ms

The #377 example, 101 x 99 at density 0.01, columns 5:11: view within 10% of the copy for sum, maximum and count along either dimension, versus the element-wise fallback before.

Tests compare every reduction with the dense result over shapes, densities and dims, for the matrix, its column-range view and with sparse = true, real and complex; init; the stored pattern; element types for small integers and Bool; empty dimensions; an allocation bound on the hypersparse path; @which for the view kernels; and the errors. Full suite, whitespace and Aqua pass locally on 1.14-DEV. Docs: a paragraph in docs/src/index.md. Not for backport.

Left out. Two pre-existing corners: fill!(spzeros(0, n), x) with x != 0 throws in _fillnonzero!, and minimum(spzeros(3, 0); dims = 1) returns a sparse 1 x 0 result through Base's map.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DGaTC2P39YrE2aAU5es55x
https://claude.ai/code/session_01SgWK99c6dYwt3gxH44MxCs

…atrix

Fixes #43. `sum(A; dims)` and the other dimensional reductions of a
`SparseMatrixCSC` returned a dense `Matrix`, which for a hypersparse
matrix costs O(size) time and memory for a result with a handful of
entries. They now return a `SparseMatrixCSC` of the reduced shape that
stores an entry only for the rows or columns that store one themselves,
unless the reduction of a structurally empty slice is nonzero, in which
case the result is fully stored. Results without a `zero`, such as the
tuples of `extrema`, stay dense.

`reducedim_initarray` provides a structurally empty destination when the
initial value is zero and a fully stored one otherwise. A new
`_mapreducedim!` for a sparse destination reduces a fully stored one as
the dense array its values form, fills an empty one without touching the
slices that store nothing, and sends anything in between through the
element-wise kernel. Row reductions build the `1 x n` result column by
column. Column reductions use the result's value vector as a dense
workspace compressed in place when there are enough stored entries, and
otherwise sort the stored entries by row so that only rows storing
something are visited.

Measured on nightly against main, min of 9: `sum(A; dims=2)` for a
10^7 x 150 matrix with 151 entries goes from 6.5 ms and 156 MiB to
2.3 µs and 16 KiB; for 10^4 x 10^4 at 1e-3 and 1e-2 density the
reductions are within noise, with the result's extra index vectors as
the only added allocation.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V6EdE4F3CCGE3vxr8gQYKf
@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.24812% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.73%. Comparing base (59bbb55) to head (996c6a2).
⚠️ Report is 31 commits behind head on main.

Files with missing lines Patch % Lines
src/sparsematrix.jl 99.24% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #798      +/-   ##
==========================================
+ Coverage   92.53%   92.73%   +0.19%     
==========================================
  Files          12       12              
  Lines        8404     8819     +415     
==========================================
+ Hits         7777     8178     +401     
- Misses        627      641      +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ViralBShah
ViralBShah marked this pull request as draft September 9, 2026 14:50
@ViralBShah
ViralBShah marked this pull request as ready for review September 10, 2026 09:25
Fixes #377.

`sum(view(A, :, j:k); dims)` and the other reductions of a column-range
view went through Base's element-wise fallback, indexing the parent once
per element. The reduction kernels only need the column pointers, row
indices and values, which a `SparseMatrixCSCView` already exposes off
the parent's storage, so they now accept `SparseMatrixCSCUnion`. The two
fully-stored fast paths index through the column pointers rather than
assuming the values start at one, and the hypersparse column reduction
sorts the view's stored range. `nnz` of such a view is now O(1).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DzbyurS9gH2pdqWJrWaPxD
@rasmushenningsson

Copy link
Copy Markdown

I'm coming from a field where the sparse matrices are not hyper-sparse and this obviously feels like a strange default to me.

The only case where I see a big benefit of the proposal is when you have many more rows than columns, very low density, and reduce with dims=2. In all other cases a dense output is the natural choice because when we lose when dimension, we have orders of magnitude fewer elements, but the fraction of non-zeros easily goes to 1. And for downstream operations, you don't want to end up with sparse arrays used as input when the data is dense.

Instead, the natural choice to me would be to opt-in to this behavior, because in a limited number of situations, it is super useful.

@ViralBShah

Copy link
Copy Markdown
Member Author

Agree - and the CSC storage format is not a good fit for hypersparse anyways.

Reductions along a dimension of a sparse matrix return a dense `Matrix` again, as
before this PR and as for dense input: the result has one dimension fewer and is
usually dense, and downstream code expects it dense. The sparse result is now
opt-in by reducing into a sparse destination, `sum!(spzeros(size(A, 1), 1), A)` or
`Base.mapreducedim!` and the other in-place reductions, which keeps the hypersparse
kernels and their cost proportional to the stored entries plus the length of the
result. A destination that stores only zeros, as a reused `sum!` destination does
after its `fill!`, folds like an empty one so that reuse stays on the fast path.
Column-range views keep reducing off the parent's storage and now return the same
dense result as their copy, where Base's `similar` gave them a sparse one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DGaTC2P39YrE2aAU5es55x
@ViralBShah ViralBShah changed the title Return sparse results from reductions along a dimension of a sparse matrix Opt-in sparse results for reductions along a dimension; reduce column-range views through the sparse kernels Sep 11, 2026
@ViralBShah

ViralBShah commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Made the sparse result opt-in, as suggested. sum(A; dims) and friends return a dense Matrix again. The sparse result is a keyword: sum(A; dims = 2, sparse = true), and likewise for prod, maximum, minimum, count, any, all and mapreduce. That keeps the hypersparse kernels for the cases where they pay off (the 10^7-row example goes from 15 ms and 153 MiB to 2.4 µs) without changing the default anyone relies on. Column-range views now return the same dense result as their copy, where Base's similar gave them a sparse one before. Description updated.

…e destination

`sum(A; dims = 2, sparse = true)` and the other reductions along a dimension
now return a `SparseMatrixCSC`; reducing into a sparse destination is no
longer special-cased. Base forwards unknown keywords from `sum`, `prod`,
`maximum`, `minimum` and `extrema` to `mapreduce`, so one `mapreduce` method
on the sparse types carries the keyword; `any`, `all` and `count` do not
forward it and get their own methods.

Without a destination the kernels can no longer rely on `sum!` and friends
having initialized it, so each slice is now seeded the way Base seeds the
dense result: with `init` when given, otherwise with `mapreduce_first` of its
first stored entry, and the unstored entries folded in afterwards. The
element type and the value of a slice with nothing to reduce come from Base's
`reducedim_init` on a stand-in, so they match the dense result exactly,
including the widening of small integers and the error for `maximum` over an
empty axis.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SgWK99c6dYwt3gxH44MxCs
@ViralBShah ViralBShah changed the title Opt-in sparse results for reductions along a dimension; reduce column-range views through the sparse kernels Opt-in sparse results for reductions along a dimension with sparse = true; reduce column-range views through the sparse kernels Sep 16, 2026
@ViralBShah ViralBShah changed the title Opt-in sparse results for reductions along a dimension with sparse = true; reduce column-range views through the sparse kernels Add a sparse keyword to reductions along a dimension and reduce column-range views through the sparse kernels Sep 17, 2026
…trim the tests

`extrema(A; dims, sparse = true)` failed with a MethodError on `zero(Tuple)`;
it now throws an ArgumentError. `any` and `all` with `sparse = true` accept
callables that are not `Function`s. The reduction grid in the tests drops
three pairs that exercise no new path, and gains complex, callable and
`extrema` cases.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New method for sum for SubArray of SparseMatrix sum(sparse) -> dense?

2 participants